It is known in several fields of computing, for example of the fields of wearable computers, and for computer user interfaces, that control of computers or computer activated devices can be achieved by monitoring human eye movements, including eye gaze direction.
Prior art eye tracking devices are known in the fields of psychological research. One such known device comprises a white screen having a predetermined pattern of dots, which a user views. An eye tracking device having a camera sensor tracks the user's eye movements when looking at the screen in front of the user, in order to calibrate the positioning of the eye tracking device and the user's eye within a three dimensional external coordinate system. However, such known eye tracking devices and calibration systems tend to be bulky and cumbersome, and typically, the camera sensor is positioned away from the user at a fixed position in a room.
Many conventional eye tracking devices use reference objects within a scene, that is within a field of view of a user in order to perform calibration of an eye tracking device. In general, a tracking device, such as a camera, can be placed in fixed relationship to an eye for which it collects tracking data, but the position of the eye with reference to the environment, or the position of the camera with reference to the environment will initially be unknown. For a tracking device and an eye in a three dimensional coordinate system, a calibration needs to be made to enable placement of the eye within the coordinate system, and to enable placement of the eye tracking device within the coordinate system.
A concise overview of eye tracking systems can be found in “Eye Controlled Media: Present and Future State”, Arne John Glenstrup and Theo Engell-Neilson, published by the University of Copenhagen DIKU (Institute of Computer Science) June 1995, viewable at www.diku.dk/˜panic/eyegaze/article, and which is incorporated herein by reference.
However, one of the well known problems in using eye motion for controlling wearable computers, wearable cameras, user interfaces and the like, is that calibration of the devices is difficult. The “Eye Controlled Media: Present and Future State” publication referred to herein above lists several problem areas concerned with eye gaze tracking techniques. Known eye gaze tracking techniques suffer from problems concerning head movement, over-sensitive tracking sensors, and equipment that loses calibration easily and quickly.
Several known calibration techniques are used for eye tracking systems as follows:
Local user initiated recalibration: A user makes local recalibration of an eye tracker system by manually moving a mouse pointer across a screen. The user stares at the pointer whilst clicking on the mouse, causing all eye gazes recorded on the vicinity of the point to be calibrated as gazes at the actual point. This calibration system has been used for a corneal/pupil reflection eye tracker.
Local automatic recalibration: This technique is based on the assumption that an “eyecon” having a small black pupil is such an attractive object that a user would not normally look at a point outside the border of the eyecon, but rather straight at the pupil of an eye. When an eye gaze is detected, either outside the eyecon or inside the eyecon, the system performs an automatic recalibration based upon a current tracking data and position of the eyecon assuming a user is looking directly at the pupil of the eyecon.
Reassignment of off-target fixations: Eye gaze fixations which are “reasonably” close to one object and “reasonably” further from all other objects, i.e., not halfway between two objects, are accepted as being on target.
Tracking data tokenization: Raw eye tracking data often contains entirely wrong coordinates, because the tracker has missed a video frame, or a user has blinked, causing discontinuity in data. In a known technique, a series of fixations separated by fast saccades are anticipated, and raw data is fitted to this expected data. Momentary spikes in raw data are interpreted as faulty measurements. A mean gaze position is reported after a short interval (100 ms) is reported as a fixation. The resulting data comprises a string of tokens which describe fixations closer to what the user thinks he/she is fixating, rather than the raw data itself.
Selection prediction using Markov-Chains: A selection prediction algorithm predicts a user's most likely next choice of object in a menu structure using a second order Markov-Chain. The prediction is used to compose a menu that is to be displayed, resulting in a dynamic menu layout system.
Wide angle for locating, tele-lens for tracking: The problems of heavy restrictions on head movement during eye tracking has been addressed by Applied Science Laboratories using an “extended head tracking unit”. A system operates simultaneously with two cameras, one with a tele-lens for eye tracking, and one with a wide angle lens to constantly locate and adjust to the user's eye position. One of the cameras locates all the faces in the field of view of a wide angle camera and selects the closest face. This face is continuously tracked using techniques for face color, e.g., skin color, and movement detection, using an artificial neural network which detects the shape of faces. General tracking of face position is combined with specific tracking of eye direction.
Combining tracking from several modalities: In this known technique, data from multiple modes is combined to resolve ambiguities in tracking data. For example, a combination of visual (face) tracking system data and speech recognition system data that is able to “listen” in specific directions greatly improves speech recognition in noisy environments.
Multi-resolution screens for speedy display response: Data in a direction in which a viewer is looking is transmitted with higher resolution than data offset from the direction in which a viewer is looking. The resolution distribution of a transmitted image is dynamically altered accordingly so that a viewer has the impression of looking at a uniformly high resolution image as they scan the image.
General known problems associated with developing eye gaze systems include:                Size and bulkiness of equipment;        Eye tracking equipment is over-sensitive to movement of a user;        Eye tracking equipment currently requires constant user attended recalibration;        Eye tracking equipment must be able to track several persons simultaneously because people work in groups together;        The problem of iris pattern recognition is not solved, and therefore current known eye tracking equipment cannot identify tracked persons;        Eye tracking equipment is not currently personalized enough to store preferences and characteristics of individual user persons.        
In many applications having direct commercial potential, such as wearable cameras, users of systems cannot be expected to perform complex calibration methods, but rather to make such applications commercially viable, calibration of eye tracking systems needs to be made simple and automatic, with little or no user input. Consequently, known calibration techniques are difficult to apply to commercially viable products which rely on eye tracking for their control or operation.
Whilst known devices are suitable for laboratory use, they are in general not suited to general mass market consumer applications. In general, they are not easily user wearable, require long training times, and are difficult to calibrate.